Skip to content

fix(reporting): suppress unchanged restatement revisions - #1918

Open
rachitm022 wants to merge 3 commits into
mainfrom
rmatani/ai-22267-suppress-unchanged-reliable-reporting
Open

rachitm022 wants to merge 3 commits into
mainfrom
rmatani/ai-22267-suppress-unchanged-reliable-reporting

Conversation

@rachitm022

Copy link
Copy Markdown
Collaborator

Summary

  • atomically reuse the readable current snapshot when a scheduled observation has the same verified content fingerprint
  • advance the observation/checkpoint without writing a revision, change-feed record, or notification event
  • preserve changed-content restatements, unreadable repairs, direct publications, and explicit official closes
  • add memory/PostgreSQL-parametrized lifecycle, replay-integrity, race, restart, and finality coverage

Finality contract

The snapshot-only Reliable Reporting 1.2 offering added by ESA #1263 does not require an official close. Its offering/configuration selects snapshot as the required finality; finality_policies governs official revisions when official finality is selected, but does not promote a snapshot-only feed into an official obligation. The final inclusive snapshot at the restatement boundary ends polling without changing finality.

There is therefore no contract mismatch to work around in the SDK. Explicit official offerings with official_close_lag continue through the authoritative close path.

Verification

  • uv run -m pytest ... — 173 passed, 71 skipped (PostgreSQL cases skip locally without ADCP_PG_TEST_URL)
  • Ruff format/check — passed
  • mypy for all changed production modules — passed
  • generated-code validation — passed
  • independent code review — ready to merge, no remaining findings

Closes AI-22267

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Nov 16, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — clean, well-tested change with no blocking or medium findings.

This PR (fix(reporting): suppress unchanged restatement revisions) atomically reuses the readable current snapshot when a scheduled observation carries the same source-manifest fingerprint, advancing the observation/checkpoint without emitting a revision, change-feed record, or notification. The reviewer verified:

  • In-memory (store.py) and PostgreSQL (pg.py) paths are behaviorally consistent across the fresh-commit suppression branch and the suppressed-replay branch.
  • The _require_current_leaf leaf-currency guard (SUPERSEDES_STALE under the account lock), the validate_revision_rows_binding content check (REVISION_CONTENT_MISMATCH, fail-closed), and the unreadable-predecessor repair fallback are all correct and covered.
  • producer.py:1878 correctly suppresses the revisions_committed signal only when the committed id differs from the intended revision.
  • No staged-upload orphan on the hot unchanged-poll path (identical content shares via find_shareable rather than re-uploading).

Gate checks: high_risk is false; gated_paths is false (so the REVIEW_REQUIRED status does not trigger row 2); no no-auto-approve team match; no prior decision. No critical/high/medium findings. None of decision-table rows 1–8 fire, so the outcome falls through to row 9 = approve.

aao-secretariat[bot]
aao-secretariat Bot previously approved these changes Nov 17, 2026

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve.

The reviewer found no critical, high, or medium findings. This PR drops the UNIQUE constraint on reporting_provisional_observations.reporting_revision_id, which is required to support the already-approved reuse-of-predecessor logic that makes observations many-to-one with revisions.

What I checked:

  • Migration correctness: fresh installs drop the constraint inline; upgrades use DROP CONSTRAINT IF EXISTS on the default <table>_<col>_key name (60 chars, no PostgreSQL identifier truncation). The change is relaxing-only and idempotent.
  • No index regression: no hot query filters observations by reporting_revision_id (PK lookup only), so removing the uniqueness does not degrade a load-bearing query path.
  • Consistency: manifest counts (27→25), schema JSON, and tests all track consistently, and an upgrade-path test was added for the new branch.

Decision-table walk-through: no critical/high findings (row 1 n/a); gated_paths is false (row 2 n/a); high_risk is false, no deletions in sensitive paths (rows 3/5 n/a); no medium findings in data-loss/schema/infra (row 4 n/a); prior decision was approve, not escalate (row 6 n/a); no no-auto-approve team match (row 7 n/a); zero medium findings (row 8 n/a). Falls through to row 9 → approve.

Note: review_decision is REVIEW_REQUIRED, but since gated_paths is false, row 2 does not apply and this does not block approval.

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — subsequent pass, test-only delta, no findings.

The delta since the prior (clean) approve is confined to test files (test_reporting_core_lifecycle.py, test_reporting_provisional_catalog_scope.py). Both tests correctly track the already-reviewed behavior: unchanged snapshot re-reads reuse the readable predecessor revision instead of emitting a new one, and the provisional-observations table drops its reporting_revision_id UNIQUE constraint. Assertions are internally consistent (27→25 schema objects, 794→792 rows, revision reuse + checkpoint advancement).

No critical/high/medium findings in this run. Checked the decision table top-down:

  • No no-auto-approve team match.
  • gated_paths: false, so row 2 does not apply despite review_decision: REVIEW_REQUIRED.
  • high_risk: false — rows 3/5 do not apply.
  • No findings of any severity — rows 1, 4, 6, 8 do not apply.
  • Prior decision was approve, so sticky-escalation row 6 does not apply.

Falls through to row 9 → approve.

@rachitm022
rachitm022 force-pushed the rmatani/ai-22267-suppress-unchanged-reliable-reporting branch from 15260f6 to 121835d Compare November 17, 2026 07:17

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve.

Subsequent pass on #1918 (unchanged snapshots reuse the readable current revision instead of emitting a new one). No critical, high, or medium findings in this run, and the prior pass was clean and still holds.

Checked:

  • validate_revision_rows_binding (store.py:703) fail-closes with REVISION_CONTENT_MISMATCH when a matching source_manifest_sha256 hides differing rows; recompute binds the predecessor's own id/totals (producer.py:228), so no spurious fire on genuine unchanged reads.
  • Replay-reuse branch (pg.py:1849, store.py:1328) and new-commit reuse branch (pg.py:1880) both gate on _require_current_leaf under the account lock (SUPERSEDES_STALE, pg.py:1690); reused ids correctly stop being appended to revisions_committed.
  • Dropping the reporting_revision_id UNIQUE constraint is safe; upgrade path covered by test_observation_schema_upgrade_removes_revision_uniqueness.
  • Race, replay-integrity, and finality coverage added. fix: prefix is correct — behavioral change, no public type-signature or wire-shape break.

Gate check: gated_paths is false, no high-risk flag, no no-auto-approve team match. review_decision is REVIEW_REQUIRED but row 2 only fires when gated_paths is true, which it is not. No blocking triggers (rows 1–8) fire; falls through to row 9 → approve.

@aao-secretariat aao-secretariat Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ladon verdict: Approve

Approve — reporting ledger snapshot reuse change is clean.

This is a subsequent pass; the prior decision was also approve with no findings, and this run again surfaces zero critical/high/medium findings.

What I checked:

  • pg.py / store.py reuse/replay blocks are fail-closed: _require_current_leaf gates the superseded-leaf invariant (SUPERSEDES_UNKNOWN/STALE), validate_revision_rows_binding recomputes the content hash under the predecessor's own id and rejects divergent rows (REVISION_CONTENT_MISMATCH), and commit_revision short-circuits on the existing predecessor before the leaf re-check so reuse cannot self-trip STALE.
  • producer.py records only genuinely-new revisions in revisions_committed.
  • The SQL UNIQUE-constraint drop is idempotent; catalog counts (27→25) and the upgrade test were updated.
  • All three race/replay branches have explicit test coverage.

Gate checks: gated_paths is false, high_risk is false, no no-auto-approve team match. review_decision is REVIEW_REQUIRED, but that only forces escalation under row 2 when gated_paths is true — it is false here, so this falls through to row 9.

No decision-table trigger (rows 1–8) fires → approve.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant